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Abstract — The Bienenstock-Cooper-Munro (BCM) and Spike 
Timing-Dependent Plasticity (STDP) rules are two experimentally 
verified form of synaptic plasticity where the alteration of synap- 
tic weight depends upon the rate and the timing of pre- and post- 
synaptic firing of action potentials, respectively. Previous studies 
have reported that under specific conditions, i.e. when a random 
train of Poissonian distributed spikes are used as inputs, and 
weight changes occur according to STDP, it has been shown that 
the BCM rule is an emergent property. Here, the applied STDP 
rule can be either classical pair-based STDP rule, or the more 
powerful triplet-based STDP rule. In this paper, we demonstrate 
the use of two distinct VLSI circuit implementations of STDP 
to examine whether BCM learning is an emergent property of 
STDP. These circuits are stimulated with random Poissonian 
spike trains. The first circuit implements the classical pair-based 
STDP, while the second circuit realizes a previously described 
triplet-based STDP rule. These two circuits are simulated using 
0.35 /im CMOS standard model in HSpice simulator. Simulation 
results demonstrate that the proposed triplet-based STDP circuit 
significantly produces the threshold-based behaviour of the BCM. 
Also, the results testify to similar behaviour for the VLSI circuit 
for pair-based STDP in generating the BCM. 

I. Introduction 

Synaptic plasticity is the study of how synapses (and the 
connection between neurons) change their functional or struc- 
tural properties in an activity-dependent manner This change 
is believed to be due to some set of underlying molecular 
processes and has been intensively studied in the last two 
decades [1|-|12|. Functional changes in synapses typically 
give rise to alterations (increase or decrease) in the num- 
ber of Q!-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid 
(AMPA) receptors. Attempts to understand how such changes 
influence network behaviour required the construction of quan- 
titative, but simplified mathematical models, which capture the 
essential nature of such plasticity processes. Understanding 
how these underlying molecular processes give rise to learning 
is believed to be one of the most important pieces of the neural 
learning and memory puzzles. There are two major classes 
of biologically inspired models for synaptic plasticity. One 
class only considers the timing difference between pre- and 
post-synaptic action potentials ||7|, | [T3| , | [T4| , while the other 
considers rate of pre-synaptic inputs and post-synaptic action 
potentials fTSl, fTS^. These classes of model have mainly 
been studied from the theoretical/computational point of view 
and use various degrees of model complexity, ranging from 
simple models using integrate-and-fire neurons |TT| through 



to investigations studying the impact of synaptic plasticity, 
including STDP, on the spatial arrangement of synaptic inputs 
on the dendrites of spatially extended neurons [17J-|20J. 
Significantly, some researchers have been physically realizing 
them as VLSI circuits to provide learning components for 
physical neural networks [21J-[26J. 

In addition to these major works, a Spike-Driven Synaptic 
Plasticity (SDSP) rule was developed to address the shortcom- 
ings of the pair-based STDP rule to faithfully learn patterns of 
mean firing rate. The SDSP rule employs both the timing of the 
pre-synaptic and the rate of the post-synaptic action potentials 
to induce synaptic weight changes [22 J , 1 ,27 1 . This rule can 
be considered as a hybrid rule which does not fall in either 
of the classes mentioned above. Here, we show that triplet- 
based STDP can give rise to the BCM rule which is a rate- 
based rule. In addition, according to a very recent study | [28) , 
triplet-based STDP can generalize the BCM rule in a way that 
it can induce selectivity with rate-based patterns with mean 
firing rates. Therefore, contrary to pair-based STDP, triplet- 
based STDP can be used to classify complex patterns and it 
is also capable of reproducing several biological experiments 
including (i) those that consider higher order spike trains (e.g. 
triplets and quadruplets of spikes) |[8), |[TT], | [T4) , and (ii) 
those that in addition to timing difference between pairs of 
spikes, bring the rate of spike-pairs into action of changing the 
synaptic weight |iO|. However, to the best of our knowledge 
the link between SDSP and its ability of reproducing the 
outcomes of these biological experiments is not established 
or demonstrated. 

In this paper, we present a neuromorphic implementation, 
where a rate-based BCM rule emerges from a VLSI circuit 
implementation of triplet-based STDP. This circuit implements 
a time-based model of synaptic change where synaptic weights 
are altered using the timing difference between pre- and post- 
synaptic spikes; however, when periods of random pre- and 
post-synaptic activities are considered, then it can exhibit the 
behaviour of a rate-based model; in this current study BCM 
(a rate-based learning rule) is an emergent property. 

The remainder of the paper is organized as follows. Sec- 
tion [n] provides a brief introduction to STDP (time-based 
model of synaptic plasticity) as well as BCM (rate-based 
model of synaptic plasticity) synaptic modification rules, and 



discusses how they are related. Section III describes the circuit 
implementations of the proposed triplet-based STDP circuit 



alongside a previously proposed pair-based STDP circuit. 
Applied protocol and circuit experimental results are demon- 
strated in Section IV Followed by a conclusion in section \V\ 



II. Synaptic modification rules 

As already mentioned there are two major classes of synap- 
tic modification rules. STDP rules modify synaptic weights 
according to the timing difference between occurrences of 
presynaptic/postsynaptic spikes |7|. One may consider just the 
timing of a pair of pre- and postsynaptic spikes | jlT| , while 
one may take the effect of a triplet/quadruplet or higher order 
sets of spike timing patterns into account |j9|, p4) . Another 
important synaptic modification rule is the BCM rule which 
changes the syanptic strength according to the rate of pre- 
and post-synaptic action potentials. In the following, firstly a 
brief review of two known STDP models is provided, then 
the BCM learning rule is introduced, and finally the relation 
between STDP and the BCM is discussed |[TTJ. 

A. Pair-based STDP 

Pair-based rule is the classical description of STDP, which 
has been widely used in various studies as well as many 
physical VLSI implementations [21 J, |24), p5[, p9J. Eg. 1 is a 
mathematical representation of the pair-based STDP rule pTj . 
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where A< — ipost ~ ^prc is the time difference between a 
single pair of pre- and post-synaptic spikes. According to this 
model, synaptic weight will be potentiated if a postsynap- 
tic spike arrives in a specified time window (t^) after the 
occurrence of a presynaptic spike. Analogously, depression 
will occur if a presynaptic spike takes place in a particular 
time window (t_) after a postsynaptic spike. The amount of 
potentiation/depression will be determined as a function of the 
timing difference between pre- and post-synaptic spikes and 
their relevant amplitude parameters (A+ and A^). 

B. Triplet-based STDP 

In this model of synaptic plasticity, synaptic weight changes 
based on the timing of a triplet combination of spikes (e.g. 
pre-post-pre, or a post-pre-post) |14|. The mathematical rep- 
resentation of this learning rule, as shown in p4) , is given 
by 
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iprc(n-i) — are time difference between combinations of pre- 
and post-synaptic spikes, e is a small positive constant which 
ensures that the weight update happens at the right time, and 
finally r_, t+, and Ty are time constants p4). 



In triplet-based STDP, potentiation can occur when a post- 
synaptic spike is generated. When this post-synaptic spike 
occurs, the synaptic weight will be increased by means of 
two different interactions. First is the interaction between 
current post-synaptic spike and its previous pre-synaptic spike 
by an amount proportional to A^. Second is the interaction 
of the current post-synaptic spike and its preceding post- 
synaptic spike by an amount proportional to A'^ . An analogous 
description holds for synaptic depression, i.e. depression is 
the result of a pre-synaptic pulse after a post-synaptic pulse 
(amplitude A^), and also a pre-synaptic pulse after a pre- 
synaptic pulse (amplitude A^). It is worth mentioning that, 
the first potentiation/depression term has a direct effect on its 
relative second potentiation/depression terms (see Eq. 2) that 
leads to nonlinearity in triplet-based STDP. 

Previous studies have illustrated that classical pair-based 
STDP fails to reproduce the experimental outcomes involving 
higher order spike patterns such as spike triplets and quadru- 
plets ||9|, p4) , 130) and, furthermore, fails to account for 
the observed dependence on repetition frequency of pairs of 



spikes |[T0|, |[T4), |j30|. Triplet-based STDP was developed to 
resolve the shortcomings of the pair-based STDP fTO) , p4) . 

C. The BCM rule 

Unlike STDP which is spike-timing based learning rule, 
synaptic modifications resulting from the BCM rule depends 
on the rate of the pre- and post-synaptic spikes | ,16J . In fact, 
it depends linearly on the pre-synaptic, but nonlinearly on the 
post-synaptic spike rate. The mathematical model of the BCM 
learning rule has been demonstrated in different ways, but a 
general, yet simple form of this model is given by [ [14) 
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AI 
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where and py are the pre-synaptic and post-synaptic spike 
rates and is a constant which represents some threshold 1 14]. 
In addition, when (f){py < 6,6) < synaptic weights will be 
decreased (depression), and when (pipy > 6,6) > 0, they will 
be increased (potentiation) and if (f>{0, 6) — 0, there will be 
no change in synaptic weight p4) . 

D. Relation of STDP to the BCM 

According to the literature, the BCM rule can emerge from 
pair-based and triplet-based STDP rules. In 2003, Izhikevich 
and Desai pT) demonstrated that, the nearest-spike interaction 

version of pair-based STDP can replicate BCM behaviour 
Furthermore, Pfister and Gerstner, in 2006 |jT4) have reported, 
a triplet-based model of STDP that can also produce BCM 
behaviour, when long-time spike statistics are taken into ac- 
count. According to pT) , under the assumption of Poissonian 
distribution of spike times for pre-synaptic and post-synaptic 
spike trains, nearest-spike pair-based STDP can give rise to 
the BCM rule; i.e. BCM emerges from nearest neighbour 

'Nearest-spike model considers the interaction of a spike only with its two 
immediate succeeding and immediate preceding nearest neighbours. 



pair-based STDP; while all-to-alj^ spike interaction cannot. 
Furthermore, based on (14], if the pre-synaptic and post- 
synaptic spike trains in a triplet-based STDP model are Poisso- 
nian spike trains, then BCM learning is an emergent property 
of the model. In the next section, VLSI implementations of 
nearest-spike interaction for both pair-based and triplet-based 
STDP are presented. Then, in the following section, adopting 
a Poissonian protocol, these circuits are tested to verify the 
emergent relationship between (nearest-spike) STDP and BCM 
learning. It is worth mentioning that, during all experiments 
in this paper, only nearest spike interactions are considered. 

III. STDP VLSI Implementation 

A. VLSI Implementation of Pair-based STDP 

There are several VLSI implementations of a pair-based 
STDP in the literature l|2T|, ||54|, ||25|, 1^, where the im- 
plementation by Indiveri et al. pT] was adopted, due to its 
low power and small area. Fig.^Ha) depicts the schematic 
circuit diagram for Indiveri's pair-based STDP circuit and 
Fig. [TJb) demonstrates its resulting temporal learning window 
for various t_|_ and r_ {Vtp,Vtd) based on our simulations. In 
this circuit, the timing of pre- and post-synaptic spikes are used 
to induce weight changes across a load capacitor, Cw When a 
presynaptic pulse, Vpio, or a postsynaptic pulse (Fpost) occurs, 
^pot (Vdcp) will be set to zero (Vkd)- This VpotCVdop) then 
changes linearly over time to reach Vdd (zero), and represents 
the required time constants r_|_(T_). These time constants can 
be set by changing the gate voltage of the corresponding 
transistors, i.e. Vtp (Vtd)- If a V^re (^^post) pulse occurs during 
the time determined by its corresponding time constant, t_ 
(t+), the output capacitor will be discharged (charged) by 
a current that is proportional to the value of Vdop (Vpot) 31 
that time and also Va- (Va+)- For further details please refer 

to nig. 

B. VLSI Implementation of Triplet-based STDP 

UnUke the pair-based model; synaptic change in the triplet 
model is induced from patterns consisting of three consecutive 
spikes, where the pattern consists of two pre-synaptic and one 
post-synaptic or two post-synaptic and one pre-synaptic spikes. 
The proposed triplet-based STDP circuit, which is configured 
to consider the spike triplet interactions in generating Long 
Term Potentiation (LTP), or Long Term Depression (LTD), is 
an extension to the pair-based circuit introduced in |21|. In 
the proposed tiiplet circuit, two more pulses, Vpost(n-i) and 
Vpre{n-i), are used in addition to Vpost(n) and Vp^^in)^ as 
shown in Fig. [2] 

This circuit operation can be described as follows: upon 
the arrival of a post-synaptic pulse, Vpast{n)^ the M5, MIO 
and Ml 8 transistor switches turn on. Then MIO sets a de- 
potentiating voltage Vdcpi to Vdd- This voltage then starts 
decaying linearly in time which can result in depression, if 
a pre-synaptic pulse, Vp-cc[n) arrives during the time Vdopi is 

-All-to-all model considers the interaction of every single spike with all 
other spikes, not only with its nearest neighbours. 
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Fig. 1. VLSI implementation of classical pair-based STDP. (a) Schematic 
circuit diagram of Indiveri et al. circuit [21 j. (b) The learning window of the 
circuit based on our simulations. 



decaying to zero (r_ time constant). In this situation, C„, will 
be discharged through M7-M9 by a current that is limited by 
the M7 bias voltage (V"^-)- In contrast to MIO, which can 
result in depression after receiving a post-synaptic pulse, M5 
and M18 can lead to two different potentiations. The first one 
can occur if M5 turns on during time constant of Vpoti (''"+)• 
This potentiation will be through M4-M6 and is proportional 
to the bias voltage at M6 (V^+)- The second potentiation 
term can charge Cyj through M16 -Ml 9 and is proportional 
to if MIS is on at the required time, i.e. when Vpoti and 
V^pot2 have still kept M16 and M17 on. This is the term that 
distinguishes triplet from pair-based STDP, as there is no such 
term in pair-based STDP. Similarly, upon the arrival of a pre- 
synaptic pulse, Vpio(n)^ a potentiating voltage Vpoti is set to 
zero and starts increasing linearly in time which can result in 
potentiation when a Vpost{n) pulse arrives within the t+ time 
constant. In addition, two possible depressions proportional to 
A2 and A3 can take place, if this pre-synaptic pulse is in the 
interval area of effect of Vdopi and Vdcp2, i-e. in t_ and 
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Fig. 2. VLSI implementation of triplet-based STDP 



time constants. 

In addition to the property where BCM learning emerges 
from the triplet-based VLSI STDP circuit, this implementa- 
tion is also capable of reproducing some other challenging 
biological experiments such as the affect of pairing frequency 
of spike pairs on weight change, as well as higher-order spike 
interaction consideration 114), [32). 



IV. Applied Protocol and Circuit Experimental 
Results 

A. Poisson Protocol 

In order to directly test whether BCM learning emerges 
from the two different STDP circuits, similar experimental 
protocol to those employed in | [T4) , pTj was adopted. Under 
this protocol, the pre-synaptic and post-synaptic spike trains 
are Poissonian spike trains, with the rate of for pre- 
synaptic, and py for post-synaptic spikes. This means that 
the inter-spike intervals between spikes can be calculated 
as: Px{s) = pa;exp(— pa;s) for the pre-synaptic spikes and 
Py{s) = pyexp{—pys) for the post-synaptic spikes. Using 
this protocol, the integral of the STDP temporal learning 
window function can be (directly) mapped to the BCM rule. 
As presented in |3T|, synaptic weight alteration for a nearest- 
spike model of pair-based STDP per one presynaptic spike can 
be given by 



CiPy)^ Py 
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Moreover, under Poissonian spike statistics, nearest-spike 
triplet-based STDP rule, as well as all-to-all triplet-based 



STDP, can both be mapped to BCM |[T4|. In this paper, 
only the nearest-spike model is considered. The total synaptic 



weight change induced by nearest-spike triplet STDP model, 
can be expressed by Eq. [5 1 14|. 
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Based on the rate-based BCM rule, synaptic weight change 
is linearly dependent on p^ and nonlinearly depends on py 
(see Eq. |3jl. In order to satisfy this condition in Eq. [s] 
must be equal to zero and also p^ t^^. This is a minimal 
case of the triplet-based STDP model (please refer to |14j). 
Also, based on the BCM learning rule definition, the synaptic 
weight modification threshold is a function of postsynaptic 
activity, i.e. 9 — a (p^) where p > 1. For triplet-based STDP, 
consider the case where all-to-all interactions between triplets 
of pre- and post-synaptic spikes; it is possible to redefine 
A2 , A2 and A'^ in a way that the threshold be dependent 
on the postsynaptic firing rate, p^. However, in the nearest- 
spike model it is not possible to change these parameters in 
a way that 9 = a(p^). Although the triplet-based nearest- 
spike STDP model cannot fully satisfy the second condition 
of a BCM learning rule (the dependency of threshold on py), 
it can elicit the properties of BCM for a limited range of 
frequencies. Matlab simulation results (Fig. [3]l show how the 
threshold is modulated by controllable amplitude parameters 
(A2 , A2 and A^) for all-to-all interaction. (Please refer to 
the text and also supplementary materials of [14J). 

B. Circuit Experimental Results 

Fig. |4]depicts the simulation results for the pair-based STDP 
circuit (Fig. [TJ, under the (earlier stated) Poisson protocol. 
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Fig. 3. Matlab simulation results of the all-to-all triplet-based STDP 1 14 j as a 
BCM rule. Three different curves show synaptic weight changes according to 
thi'ee different synaptic modification thresholds which demonstrate the points 
where LTD changes to LTP. The thi'eshold is adjustable using the STDP rale 
parameters (please refer to the text). 
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Fig. 4. Simulation results of the nearest-neighbour pair-based STDP (Fig.[TJ 
as a BCM rule. Three different curves show synaptic weight changes according 
to three different synaptic modification thresholds which demonstrate the 
points where LTD changes to LTP. The threshold is adjustable using the STDP 
rule parameters (please refer to the text). 



In this circuit, V^post and Vprc are Poissonian spike trains 
where the firing rate of pre- and post-synaptic spikes are py 
and px, respectively. For the above mentioned circuit, during 
the simulations, a BCM-like behaviour emerges from nearest- 
spike pair-based STDP as demonstrated in Fig. [4] In this 
circuit the modification threshold can be calculated by the 



following (31 1, 



(6) 



For producing the three different waveforms shown in 
Fig. |4j three different thresholds have been used by keeping 
Va_, Vtp and Vtd fixed and adjusting Va+- 

As already mentioned, besides pair-based STDP, BCM 
behaviour can also be replicated by triplet-based model of 



STDP [ [T4) . Fig. |2] demonstrates the proposed VLSI implemen- 
tation of the nearest-spike triplet-based STDP rule. In order to 
take the BCM characteristics out of this STDP circuit, either 
V^- must be set to zero, or the second depression part of 
the circuit on the bottom right part of the circuit (Ml 3 -Ml 5 
and M20-M23) can be eliminated. This will be a minimal 
triplet-based STDP circuit which satisfies the first criteria of 
the BCM rule. In this circuit, ypost(n-i), ^post(n) and Fpro(n) 
are Poissonian spike trains where py, py and p^ denote their 
firing rates, respectively. Fig. |5] depicts the simulation results 
of this circuit under the Poisson protocol. Three different 
curves are presented which display three different weight 
modification thresholds. In the BCM rule, these thresholds 
are related to the post-synaptic firing rate, py. Based on | [T4) , 
the modification threshold changes for the case when all-to- 
all spike interactions are considered and can be expressed as 
Eq.|7] 



(PoAtT+Ty) 
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where (p^) is the expectation over the statistics of the p 
power of the post-synaptic firing rate, pJJ = (p^j) for large 
time constants (10 min or more). However, for the nearest- 
spike model which is the case of the proposed circuit, it is not 
possible to derive an equation for the modification threshold 
based on py, but for postsynaptic firing rate up to 50, a 
similar behaviour to what Eq. |7] presents is inferable from 
the simulation results (supplementary materials of ||14J). 

It should be mentioned that, the simulation results are 
accelerated 1000 times when compared to real biological time. 
This means that for the proposed VLSI circuit, 1 ms of time 
used in the circuit represents one second of biological time. 
Put simply, in Fig. |4] py is the average firing rate of the post- 
synaptic neuron is given by spikes per millisecond rather than 
spikes per second (here 1 ms of circuit time is equivalent to 1 
second). This accelerated approach has been used in previous 
studies of implementing STDP with VLSI such as f^, (29). 

The current circuit implementation in its present form is 
sensitive to variations in bias voltages, as it is the case with 
the original circuit by Indiveri et al. pT| . Hence, in the 
presented design, each bias voltage can be provided using a 
diode-connected MOS device in series with a current source, 
resulting in a current mirror structure. As the current values to 
achieve the needed biases are very low in the sub-nanoampere 
(nA) levels range, some of these transistors will be forced 
to operate in the subthreshold regime of operation. This 
approach will result in a more robust approach for bias voltage 
setting instead of direct biasing using fixed voltage sources. 
Furthermore, these current sources can be combined with 
digital trimming techniques to elevate some of these short 
comings due to process variations and allow for bias voltage 
adjustment even after circuit fabrication. 

To date, there have been two different proposed VLSI 
circuits which implement the BCM rule p3l, p6), 133). To 



the best of our knowledge, just the VLSI circuit in 1 23 1 
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Fig. 5. Simulation results of the nearest-neighbour triplet-based STDP 
(Fig. |2) as a BCM rule. Three different curves show synaptic weight 
changes according to three different synaptic modification thresholds which 
demonstrate the points where LTD changes to LTP. The threshold is adjustable 
using the STDP rule parameters (please refer to the text). 



has the same capabiHties as the proposed triplet-based STDP 
VLSI circuit presented in this paper. That circuit implements a 
recently introduced BCM-like model (34], which is capable of 
replicating pairing frequency effects and triplet spike effects 
(quadruplet experiments have not been shown in this paper), 
and of course BCM behaviour In comparison to the proposed 
circuit presented in this paper, the circuit proposed in | ,23J 
contains more devices, hence larger silicon area when imple- 
mented and more prone to process variation. The neuron archi- 
tecture also needs to be changed in a way that is compatible to 
the learning algorithm, while our VLSI circuit simply acts as 
a plastic synapse, which can be used to connect to other sets 
of (neuromorphic) neurons of choice. Furthermore, the results 
from our proposed VLSI implementation can reproduce the 
outcomes of previously published BCM experiments Q, Q, 
|[16J. 

V. Conclusion 

A new VLSI implementation, based upon spike-timing 
leading to the emergence of the BCM rule was presented. 
The proposed implementation is based upon a triplet-based 
nearest-spike STDP model. This model can replicate the same 
BCM behaviour as previous studies |23|, p3), as well as 



other important synaptic plasticity experiments, including the 
effects of pairing frequency of spike pairs on weight change 
and higher-order spike interactions (triplet and quadruplet). 
It was also shown that the VLSI implementation of classical 
pair-based STDP, which is based on the interaction between 
pairs of pre- and post-synaptic spikes rather than the triplets 
of spikes can also reproduce similar behaviour as the BCM 
rule. The results of this study can be a powerful addition to 
classical pair-based STDP, which is known not to be capable 
of mimicking other plasticity based experimental phenomena. 
The presented design can play an important role in hardware 
implementation of spiking-based neural networks. 
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